The present invention generally relates to distributed file systems, and more particularly to object management in distributed file systems.
In a distributed file system the data are distributed between network-connected nodes. Distributed file systems are scaleable to satisfy increasing storage and access requirements. As more storage resources are required to satisfy the requirements of various applications, new nodes with additional storage capacity can be added to the distributed file system.
The structuring of data between different nodes may impact the performance of certain applications, depending on data access patterns. For example, if one node provides access to two data sets, heavy access to the two data sets may impact the required performance level. To alleviate this type of performance bottleneck, contention for the storage resources is reduced by redistributing selected data sets amongst the different nodes.
Data are traditionally migrated at the block level, with no movement of the associated control data. Such control data includes, for example a file system directory entry, which is sometimes referred to as xe2x80x9cmeta-dataxe2x80x9d. If the control data associated with the competing data sets are not separated, the performance level may continue to be negatively impacted. Because referencing data at the block level does not indicate whether the block contains data or meta-data, it is not feasible to migrate meta-data and data blocks together.
Another approach to redistributing data is to move data at the granularity of a file system. This results in both control data and block-level data being moved. However, a file system typically includes many files that are hierarchically organized within many different directories. Thus, moving an entire file system may not separate the data sets that need to be separated. It can be seen with present systems that the granularity of moving data at the block level may be too fine, and the granularity of moving data at the file system level may be too coarse.
Migrating objects in a distributed file system also presents issues for the clients that reference the objects. One issue is the validity of cached file handles after objects have been migrated. Generally, the file handle identifies the host node and other information for referencing a file. The file handle is provided by a namespace server to a client in response to an access request, and the client caches the file handle for future references. If the data referenced by the file handle is moved, the cached file handle will be invalid. Thus, cached file handle must be revalidated for it to be useful to the client.
A system and method that address the aforementioned problems, as well as other related problems, are therefore desirable.
In various embodiments, the invention moves an object from a source partition server to a target partition server in a distributed file system having multiple partition servers. Each object has at least one associated and linked parent object, and each partition server owns a subset of objects of the file system. The object migration is accomplished by creating a copy in a target partition server of a selected object from a source partition server. At the source partition server, a forwarding link is created that references the copy in the target partition server. The copy is linked to the parent object of the selected object, and requests for access to the selected object are responded to with the forwarding link.
Various example embodiments are set forth in the Detailed Description and Claims which follow.